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Abstract 

The problem of embedding the Tsallis and Renyi entropies in the 
framework of category theory and their axiomatic foundation is studied. 
To this end, we constrnct a special category MES related to measnred 
spaces. We prove that both of the Renyi and Tsallis entropies can be 
imbedded in the formalism of category theory by proving that the same 
basic functional that appears in their definitions, as well as in the associ¬ 
ated Lebesgue space norms, has good algebraic compatibility properties. 
We prove that this functional is both additive and multiplicative with re¬ 
spect to the direct product and the disjoint sum (the coproduct) in the 
category MES, so it is a natural candidate for the measure of informa¬ 
tion or uncertainty. We prove that the category MES can be extended 
to monoidal category, both with respect to the direct product as well as 
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to the coproduct. The basic ajcioms of the original Renyi entropy theory 
are generalized and reformulated in the framework of category MES and 
we prove that these axioms foresee the existence of an universal expo¬ 
nent having the same values for all the objects of the category MES. In 
addition, this universal exponent is the parameter, which appears in the 
definition of the Tsallis and Renyi entropies. 


1 Introduction 

The discovery of two related generalizations of the classical Shannon entropy [T] 
is a remarkable coincidence in the history of abstract probability theory and sta¬ 
tistical physics. A. Renyi introduced a possible generalization [5] of the classical 
Shannon entropy by pure axiomatic extension of the Fadeev axioms 0, la that 
define uniquely the Shannon entropy. On the other hand, the generalized en¬ 
tropy 0, 0 introduced by C. Tsallis was useful to extend the classical maximum 
entropy principle such that the heavy tailed distributions observed in a large 
scale of physical processes m, 0,0, cni, could be derived from (generalized) 
maximum entropy principles. The interest in the study of the generalizations of 
the Shannon entropy in the recent years is due to the multiple applications of 
the Tsallis and Renyi entropy or the associated Renyi divergence 0,0, m, 
m- We mention also that similar to the classical H theorem of L. Boltzmann, 
the generalizations of the Renyi entropy, as well as the original Renyi entropy, 
is a Liapunov functional for a large class of stochastic processes described by 
generalized Fokker-Planck equations, more exactly by Fokker-Planck equation 
where the drift term and the diffusion tensor are itself dependent on some ex¬ 
ternal random variable m- We mention that in the case of suitable singular 
limiting procedure, both the Tsallis and Renyi entropies give the same limit: 
the Shannon entropy. The Renyi entropy is additive while the Tsallis entropy 
is not. Despite the Renyi and Tsallis entropies give the same results in the case 
of problems associated to the determination of the probability density function 
from the Maximum Entropy principles, because they are algebraically related 
by simple formulae, the non-additivity of the Tsallis entropy generated many 
discussions in the physical literature. On the other hand, by formulating the ba¬ 
sic axioms [5], A. Renyi introduced new concepts {incomplete random variables 
and incomplete distributions) that are not included in the standard terminology 
of the probability theory. Also the formulation of the Postulate 5’ [2], is not the 
simplest, mathematically natural. 

In this work we develop a formalism in the framework of the category theory 
m and m for the study of generalized entropies. The category theory is the 
branch of mathematics that plays a central role in the logical foundation and 
synthesis of the whole contemporary mathematics. In particular, the category 
theory allows avoiding the paradoxes of the classical set theory. In order to 
highlight the natural structures related to generalized entropies, we use the 
central concepts of the modern mathematics. 

The paper is organized as follows. In the Section [2l Subsection 12.11 we define 
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a special category related to measurable spaces (referred to as MES), enabling 
the introduction an associated basic functional Zp (see the forthcoming Section 
for his exact definition). Both the Tsallis and Renyi entropies, as well as the 
distance in spaces, may be expressed in terms of this functional. In the 
Subsection O we define the direct product of the objects in MES and we 
prove that the functional Zp satisfies a compatibility relation with respect to 
this product i.e., it is multiplieative. This multiplicative property is equivalent 
to the additivity of the Renyi entropy. In the Subsection 12.31 we define the dis¬ 
joint sum (or the coproduct) of the objects in MES, and we prove that the 
functional Zp satisfies a compatibility relation with respect to coproduct i.e., 
it is additive. Note that this property is equivalent to one of the postulates 
characterizing the Renyi entropy. The prove that both product and coproduct 
possess a universal property and that the direct product and coproduct can also 
be defined for morphisms of the category MES, can be found in the Subsec¬ 
tion nn In the Subsection 12.51 we show that, by extending the category MES 
with the introduction of the unit objeet and the null object, the category MES 
became to a monoidal category. 

Section [3] deals with the axiomatic characterization of the functional Zp. We 
demonstrate that there exists a universal exponent p (the same for all the objects 
of the category) that characterizes completely the functional Zp (hence, also the 
Tsallis or Renyi entropies) up to an arbitrary multiplicative factor. 

Appendix 15.II shows that the Renyi divergence can be expressed in terms of the 
Renyi entropy. The proof of the universality (with respect to all the objects of 
the category MES) of the exponent defining the Renyi or Tsallis entropies can 
be found in Appendix 15.21 


2 The category-theoretic properties related to 
Renyi and Tsallis entropies. 

2.1 Definitions 

Our definitions include as a particular case the original definition of the gener¬ 
alized entropies and [2] . Our basic construction that will play the role of 

the object of the category MES is derived from the well known concept of mea¬ 
surable space [16] , m- Guided by statistical ideas, in order to take into account 
the negligible sets we specify also an sub-ideal of the cr-algebra of measurable 
sets. The objects of the category MES consist of triplets Mx ■= {X,Ax,J^x) 
with X denoting the phase space (for instance, it is a symplectic manifold in the 
case of statistical physics or, in the case of elementary probability models, finite 
or denumerable set) and Ax is the cr—algebra generated by a family of subsets 
of X, respectively. We also denote with Afx C Ax an ideal of the cr-algebra Ax 
having the meaning of negligible sets. Let us now postulate the completeness 
property. From N S Mx and N' C N results N' S Mx ■ The morphisms of the 
category MES with the source Mx and range My are the measurable maps $ 
from X to Y , which are nonsingular i.e., such that <I>“^(A/V) C Mx- From the 
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completeness property results the ideal property i.e., if N € Mx and A G Ax 
then An N G Mx■ Note that it is possible that Nx contains only the empty 
set (as, for example, in the case of atomic spaces). 

Remark 1 At first sight it would be more natural to consider the objects as 
measure space triplet {X, Ax, yi-x) containing the measure fix, and the mor- 
phisms as the measure preserving transformations. However, in this case we 
cannot define direct product or coproduct having universal property. 

We denote with C{Mx), or with C{X,Ax,Afx), the cone with all a— finite 
positive measures over (X,Ax,J^x) that are compatible with Mx (i-e., /r G 
C(X,Ax,J^x) iff for all N G Nx we have pl{N) = 0). For a given pLx G 
C{X, Ax,Afx) andp > 0, we denote with LP{Mx, yi-x) the Banach space {p > 1) 
or the Frechet space (0 < p < 1) of functions fx-X^M. that are measurable 
modulo Afx and have finite norm (pseudo norm, respectively): more precisely, 
/ \fx{x)f dp.x{x) < oo. In the sequel, we shall denote 

X 


Zp{Mx, p-x, Px ) '■= J Px{xYdpx[x) (1) 

X 

for some non-negative density px G LP{Mx, px)- The generalized entropies are 
defined for probability density functions (PDF) satisfying the conditions 

Px G L^{Mx, Px) LP{Mx, Px)', (2) 

J px{x)dfix{x) = 1 (3) 

X 

where p > 0 and p ^ 1. The probability P{A) can be represented by PDF as 
follows 


PiA) = J pxix)dpxix)-, (4) 

A c X; A G Ax', Px G C{Mx) (5) 

In this framework, for a given measurable space Mx '.= {X,Ax,Afx) and mea¬ 
sure Px G C(Mx), the classical Boltzmann-Gibbs-Shannon entropy functional 
is given by 

Sci[Mx,px,Px] = - J'px(^) log [px(^)]dpx(^) (6) 

X 

For a given measurable space Mx, the generalizations of the A. Renyi [5] and 
C. Tsallis [5], [6] entropies, involves the functional Zp{Mx, px, Px) given by 
Eq.O. The functional Zp is related to the norm of the density p in the Banach 
space for p > 1 m. and to the pseudo-norm Np[p] for 0 < p < 1 [Hj [18], 
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through the obvious relations 


IIP^IIp 


[px{xyf dpx{x) 


Lx 


p>\ 


(7) 


Np[px] = J [pxix)f dpxix)] 0<p<l (8) 

n 

These relations give the geometrical interpretation of the generalized entropies 
(for further information Refs to [13]). 


Remark 2 The study of the generalized entropies helps us to better understand 
the classical entropy. For p > 1, the functional ||px||p is the classical norm, 
and for 0 < p < 1 the functional Np[px\ is the exotic L^-norm U8\/ . For p > 1 
the LP spaces are reflexive, the Maxent problem is equivalent to the minimal 
LP distance problem with restrictions IW, or to the minimal Zp(Mx, px, Px)- 
For 0 < p < 1, the LP spaces has, in general, trivial duals, the Maxent problem 
is equivalent to the maximal LP distance or the maximal Zp{Mx, px, Px) (see 
mj- The case p = 1, which corresponds to the classical Shannon entropy, is 
just the border point between two radically different functional-analytic proper¬ 
ties. 


The corresponding generalized entropy Sn.p, proposed by A. Renyi [2], and the 
entropy, Sr.p, proposed by C. Tsallis 0, m are given by 

Sr,p[Mx, px, Px] = j —- ^og Zp{Mx, px, Px) (9) 

St,p[Mx, px, Px] = —- [1 — Zp{Mx, Px, Px)] (10) 

Consider now a measure space N = (f2, A, n) with a-finite measure n, and let 
us denote with P{x), Q{x) two probability densities: 


P{x)dn(x.) = / Q{x)dn{ 


= 1 


Note that the Renyi divergence 0, m 

Dp{P\\Q) = ^log J PPQ^-^dnix) 

n 


( 11 ) 


is related to the Renyi entropies, Eq.®, by Ea. (H51) (see Appendix 15.ip . Note 
that when A is a finite or denumerable set, if we denote with pk the probabilities 
of element Xk G A, the measure px is the counting measure on the space 
A (equal to the number of elements in a subset), and the family of null sets 
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■A/x = {0} then, from the previous Eqs.([Hl[Tni[D) we get the original definitions 
from Ref. [2], [5],[6] 


SR,q\Mx,^J.x,Px] = pI 

^ k 

^-Y.pt 

k 

Zq{Mx,px,Px) = YPk 


Sr.qlMx, px, Px] = 


1-p 


( 12 ) 

(13) 

(14) 


Remark that, in this particular case, ST,q[Mx, px, Px], a.swe\l as Zq{Mx, px, Px), 
are Lesche stable m- Note that, from Eqs (151 l^and llOl) . results 


\iinST,q[Mx,px,Px] = ^innSR^qlMx, px, Px] = Sci[Mx, px, Px] (15) 

p —>^1 p —^1 


2.2 Direct product of measurable spaces and the multi¬ 
plicative property of Zp[Mx, Px] 

In the framework of the our formalism, the multiplicative property is the coun¬ 
terpart of the Postulate 4 in the Renyi theory [2]. In the following we over¬ 
load the tensor product notation ” 0 ”; its meaning results from the na¬ 
ture of the operand. Denote the direct product of two measurable spaces 
Mx = {X,Ax,J^x) and My = {Y,Ay,Ny) by Mx ®My^ defined as fol¬ 
lows 

Mx ® Mx = {X xY,Ax ® Ay,Mx®y) (16) 

Here X x E is the Cartesian product of the phase spaces X and Y, while the 
cr-algebra Ax ® Ay is the smallest cr-algebra such that it contains all of the 
elements of the Cartesian product Ax x Ay- The null set ideal Afx^yC Ax ® 
Ay is generated by the family {Ax Afy) U (Ax 0 Ay). Note that if px & 
C[Mx] and py & C[My] then their direct product satisfies the condition px <8) 
Py G C[Mx Z) My] (we denote it also by the same symbol). The measure 
Px Z Py acting on {Aj^ Z Ay) /J^x^y are defined by extension by denumerable 
additivity, starting from the product subsets: 

{px Z py){Ax X Ay) = px{Ax)pY{Ay) (17) 

Ax G Ax; Ay G Ay (18) 

Consider now the measures px G C{Mx), py G C{My), and the densities 
Px G L^{Mx,dpx) L^{Mx,dpx) and py G L^{MY,dpY) Cl {My, dpx)■ 
The following function is also denoted with the same symbol 

Px Z py € LP{Mx X My,px Z py) nL^{Mx x My^px Z py) (19) 

with 


{px ZpY){x,y) = px{x)pY{y) 
X eX; y eY 
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( 20 ) 

( 21 ) 






We have the following basic proposition 
Proposition 3 Let px , pv are normalized PDF 


J pxix)dfix{x) = J pYix)dfiY{y) = 1; Px >0; py >0 (22) 

X Y 

Then we have 

Zp[Mx ^ My, Px ^ h-Y, Px ^ Py] = Zp[Mx, p^x, Px] ZplMy^py^py] (23) 

Sr,p[Mx 0 My, Px 'Z py, Px 'Z py] = <S'_R.p[Mx, Px,px] + *S'_R,p[My, py, py] 

(24) 

The validity of this statement follows directly from the definitions of the direct 
product, the Renyi entropy and the functional Zp. 

2.3 Coproduct of measurable spaces and the additivity of 
the functional Zp[Mx, Px, Px] 

Let us study now the property encoded in the Postulate 5’ related to the Renyi 
entropy theory (Ref. [2]), transcribed in the measure theoretic and category 
language and re -expressed in the term of the functional Zp [Mx, px, Px] ■ Also 
in this case, we overload the notation U, for the disjoint sum from the set 
theory. Its precise meaning will be clear from the nature of the operands. In 
the following we investigate the functorial properties, related to Postulate 5’, 
of the functional Zp[Mx, px, p], in analogy to Proposition [3] To this end we 
introduce the following 

Definition 4 The coproduct of measurable spaces Mx = {X,Ax,Yx) and 
My = {Y,ZW,Yy) will be denoted by Mx^My and have the following structure 

Mx LI My = (X U Y, Ax LI Ay,Afx LI Afy) (25) 

Here, XUY is the disjoint sum of the sets X andY, and Ax'z\Ay is the smallest 
a-algebra that contains all of the sets of the form Ai U A 2 , with Ai € Ax and 
A 2 G Ay, respectively. Moreover, the new null set ideal Afx^Yy is the smallest 
a-algebra generated by the family Ni U N 2 with Ni G Mx and N 2 G My. Let 
the measures px G C(Mx), py G C(My) and the weights wi > 0, W 2 > 0 and 
wi-\-W 2 = 1. The measure p := wipx \-iw 2 py acts on the a - algebra Ax UAy 
and it is defined uniquely as the continuation by denumerable additivity from 
the property 

p{Ai) = wipx{Ai)-, Ai G Ax (26) 

p(A 2 ) = W 2 Py(A 2 ); A 2 G Ay (27) 

Let Px G LP(Mx, dpx) H L^{Mx, dpx) and py G L^[My, dpy) fl L^{My, dpy) 

. We define the function p := pxi^py G LP(MxUMy,wipxUw 2 py)nL^(MxU 
My, wipx LI W 2 py) as follows 

p{x) = px{x)-, if X G X 
p{x) = pyix); if X GY 
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We restrict our definition of coproduct to finite terms. An example of (denu¬ 
merable infinite) coproduct is the grand canonical ensemble. 

Remark 5 If pxd^x and pydpy are probability measures, then the measure 
[pi U P 2 \{x){widpx LI W 2 dpLy] is a probability measure if Wi + W 2 = I ■ 

From the previous definition of the direct sum and the functional Zp[Mx, Px, Px] 
the following obvious proposition results 

Proposition 6 The reformulation of the Postulate 5’ (Ref. reads: the func¬ 
tional Zp [Mx, px, Px] is additive with respect to the direct sum of measurable 


spaces 


ZplMx^lMy, WipxyiW 2 Py, px'^Pv] = WiZplMx, px, Px]+W 2 Zp[My, py,py] 


(28) 


2.4 Universal properties of the direct product and direct 
sum in the category of measurable spaces 

In the following we prove that the basic binary operations on measurable spaces, 
the direct product and the direct sum, defined in the previous section, have 
universality properties in the category of measurable spaces MES. 

Consider the direct product M = Mx 0 My of measurable spaces Mx = 
(X,Ax, Ax) and My = {Y,Ay,Ny). Observe that the canonical projections 
Px '■ X xY X, py : X X Y Y, are measurable and induce the morphisms 
TTx : Mx O My — Mx and wy : Mx O My —>■ My between the objects of 
MES. We have the following 

Proposition 7 In the category MES the applications ttx : Mx ® My — >■ Mx, 
xy : Mx ® My My, which are naturally induced by canonical projections 
Px : X xY ^ X and py : X xY ^Y, are morphisms. 

Proof. The measurability of ttx is direct consequence of the fact that the canon¬ 
ical projection maps are measurable, in fact the measurability of the canonical 
projections is an alternative definition of the product of a algebras. The nonsin¬ 
gularity property ^\Mx) C Nxxy results from ^\j\fx) = Ax x Ay C 
Axxv • ■ 

From the previous Proposition [3 results immediately the following Theorem 

Theorem 8 In the category MES, the direct product has the universal prop¬ 
erty. Let Mx = (A, Ax)Ax). My = (Y,Ay,Afy) and M = [Z, Az,ffz) 
measurable spaces that are objects of the category MES, such that there exists 
morphisms (fx G Hom{M,M x) o,nd ipy G Hom(M,My). Then there exists 
an unique morphism 9 G Hom{M,Mx <8> My) such that 


4>x =T^x o 9 
(j)y = TTy O 9 


(29) 

(30) 
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where ttx , t^y o-fe the morphism defined in Proposition . 

Proof. The morphism 9 is induced by the application T : Z ^ X xY defined as 
Z 5 z ^ T{z) := {(j)x{z), fiviz)) G X xY. and it is unique. In order to prove 
that 6 is a morphism we have to prove that T is measurable and it is nonsingular. 
To prove that T : Z ^ X x Y is measurable, we recall that it is sufficient to 
prove that, for all A G Ax, B G Ay , we have the property x B) € Az, 

a property resulting from the measurability of fix o-nd fiy • Note that to prove the 
inclusion T~^{J\fxxy) C Nz, it is sufficient to demonstrate for the generating 
subsets T~^{J\fx x Ay ) C Mz (which follows from the nonsingularity of fix and 
T~^{Ax X. Afy) C ffiz) that this is the consequence of the nonsingularity of fiy. 


In conclusion the direct product operation has the natural functorial property, so 
the multiplicative property Eq. (l23ll of the functional Zp(Mx, Hx, Px) appears 
as an algebraic compatibility property. By simple reversal of the arrows, we are 
lead to the corresponding universality property of the coproduct in the category 
MES. We have the following obvious proposition 

Proposition 9 In the category MES, consider the objects Mx,My. The ap¬ 
plications Lx ■ Mx — >■ Mx U My and Ly : My —^ Mx LI My, induced naturally 
by the canonical injections ix '■ X ^ X \AY, iy : Y X\AY, are morphism in 
the category MES. 

Proof. The injections ix, iy are measurable. Suppose that iViUA ^2 & A/xLlA/V, 
with 7Vi G Afx, N2 G Afy (see Definitions^. Then, i^^^\Ni 11X2) = Ni, 
U N2) = N2, so ix and iy are nonsingular, which completes the proof 
that Lx , Ly are morphisms in the category MES. ■ 

By reversing the arrows, in analogy to the Theorem |8l we obtain the following 
result 

Theorem 10 In the category MES the direct sum of the objects has the fol¬ 
lowing universality property. Let denote with Mx = {X,Ax,Afx), Aly = 
{Y,Ay,Ny) and M = [Z,Az,Mz) measurable spaces that are objects of the 
category MES, such that there exists morphisms fix G Hom{Mx, M) and fiy G 
Hcmi{My, M). Then, there exists an unique morphism 7 G Hom{Mx\-iMy, M) 
such that 


1° Lx = fix 

'y O Ly = fiy 

where lx ,lx o,re the morphisms defined in Propositions^ 

Proof. The morphism 7 is induced by the map g : X U Y —^Z defined as 
follows. If X € X then g{x) := fix{x) G Z, and in the case x € Y, then g(x) := 
fiy{x) G Z. The measurability of the map g results from the measurability affix 
and fiy. The inclusion g^~^\Mz) C Mx U A/y results from the nonsingularity 
of fix and fiy. m 
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In conclusion, the direct sum operation has natural category theoretic proper¬ 
ties. He,Hence, the additivity property Eq.([28]) of the functional Zp{Mx, ^x, Px) 
is not an artificial construction. 

2.5 The monoidal categories associated to product and 
coproduct 

We recall the following 

Proposition 11 /15] / LetC he a category such that for all objects A,Bg Ob{C) 
exists their direct product A ® B, having the universal property. Then, there 
exists a covariant functor F from the product category to C, Cx C ^ C ,defined 
as follows. For the object (A, B) ofCxC, where A, B are objects ofC, we have 

F{{A,B)) :=A(8)B 

For the pair of morphisms {u,v) G Hom{{A,B), {A',B')) withu € Hom(A, A'), 

V € F[om{B, B'), from the category C x C there exists an unique morphism w in 
the category C, w € F[om{A ® B, A' ® B') uniquely fixed by the conditions 

w = F((u, v)) 

PA' o w = u o PA 
PB' o w = V O PA 

We denoted withpA, Pb the projections from Flcmi^A® B, A), Hom{A®B,B), 
andpA' arepB' the projections from F[om{ A'®B', A'), FI om{ A'(i)B',B'). The 
map {u,v) —>■ F{{u,v)) has the functorial property. 

Let {u, v) € Hom((A, B), {A', B')) and {u', v') € Hom((A', B'), (H” ,B”)). Then, 

F{{u ou,v' ov )) = F[{u',v' ))oF{{u,v )) £ F[om{A® B,A" ® B") 

If in the category C we have an unit object, then C is a monoidal category. 

Similarly, by duality arguments, we have the following result for the direct sum 
(coproduct) 

Proposition 12 \15^ Let C be a category such that for all objects A, B from 
Ob{ C) exists their direct sum A\A B, having the universal property. Then, 
there exists a covariant functor G from the product category CxC -^C defined 
as follows. For the object (A,B) of Cx C, where A, B are objects of C we have 

G{{A,B)) —AUB 

For the pair of morphisms {u,v) G Hom{{A, B), {A', B')), withu G Hom{A, A') 
and V G Hom{B, B'), from the category CxC there exists an unique morphism 
w in the category C, w G Hom{A \A B,A' \A B') uniquely fixed by the conditions 

w = G{{u, w)) 
w o iA = iA' ° u 

w o is = is' ° V 
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We denoted with iA , is the canonical injections from Hom{A, AUB), Hom{B, AU 
B), and withiA' , is' the injections from Hom{A', A' U B'), Hom{B', A' Li B'). 
The association (u,f) ^ G{{u,v)) has the functorial property. Let (u,v) € 
Hom((A, B), (A', B'}} and {u',v') G Hom{{A',B'),{A'\B”)) then, 

G{{u'ou,v'ov )) = G{{u',v' ))oG{{u,v )) G Hom{AU B,W^ U B”) 

If in the category C we have a null object then, C is a monoidal category with 
respect to direct sum. 

We emphasize that, despite the fact that the construction of the direct sum is 
dual to the direct product, from the previous proposition (TT^ the functor G is 
a covariant functor. In the category MES we have an unit object as well as 
the null object. The unit object is denoted with Mi := where 1 

is the one point set m, Ai is the trivial cr-algebra consisting in the one point 
set 1, 0, and Mi = {0}, respectively. The (more or less formal) null object 
Mq, with respect to the direct sum, is the object generated by the empty set 
Mo := {0,A0, A/ 0 ). So we have the following 

Conclusion 13 The category MES is a monoidal category both with respect to 
the product 0 and the coproduct U. 

3 Axioms 

We expose another approach, based on category theory, to the problem of the 
naturalness of the choice of the family of functions ga used in the definition 
of the entropy [2]. We prove that this problem may be treated if we take into 
account the additivity and the multiplicative properties of the functional Zp. 
We mention that a possible candidate for the generalization of the symmetry 
Postulate 1 [ 2 ] is the requirement of invariance of the generalized entropy un¬ 
der measure preserving transformations. Recall that the group generated by 
finite permutations is the maximal measure preserving group with respect to 
the counting measure. The problem is that there are plenty of measures such 
that the measure preserving group is trivial (for instance, the atomic measure 
for 2 element set with /i(l) m( 2)). To avoid this problem, we observe that 

Postulate 1 and Postulate 5’ in the original Renyi theory [5] can be generalized 
as follows. For a given measurable function f{x) on the measured space Mx 
and p. G G{Mx), let us define 

mf(Mx,p,t) = p [{x\x gX k f{x) < t)] (31) 

Note that mf{Mx,p,t) is invariant under measure preserving transformations. 

In addition 

00 

Zp[Mx,px,p] = j Pdmp{Mx,p,t) (32) 

0 


II 


Then, the Postulate 1 (the symmetry property) and Postulate 5’ (the additivity 
property expressed in Proposition [6]) can be generalized as follows. Postulate 1 
& Postulate 5’ 

OO 

Zp[Mx,tix,Px] = j hxit) dmp^{Mx,p,t) = j hx [px{x)\dp,x{x) (33) 

0 X 

hx{x) > 0] if a: > 0 (34) 

for some Borel measurable function hx(t) with 

hxiO) = 0 (35) 

The last requirement result by considering the case when the support of px is 
concentrated on a proper subset of X and by using Ea. (E51) . The generaliza¬ 
tion of the Postulate 2 (the continuity property) is straightforward. Be h{x) 
continues and px S L^{Mx, Px), we get 

hx [p{.x)] e L^{Mx,px) (36) 

In our settings, the analog of the Postulate 4 (the additivity property) [2] is the 
multiplicative property given by Ea. (|23)) and Proposition |3l By using Eqs.([23l 
[33l [35] and [36|) , and by continuity of the functions hxY, hx, hy for all x,y >0, 
we obtain the following functional equation (valid almost everywhere) 

hxvix y) = hx{x)hY{y)-, a;,?/ e R (37) 

By arguments similar to the proof of the uniqueness, from Theorem 2 m) , we 
get Eq. (1321) (for details see the Appendix 15.21) : there exists an universal family 
of functions, independent of X , parametrized by the positive parameter p such 
that 

hx{x)=xPCx (38) 

hriy) = yPCy (39) 

hxviz) = zPCxCy (40) 


4 Summary and conclusions 

We proved that the most natural setting for treating the axiomatic approach to 
the study of definitions of measures of information or uncertainty, is the formal¬ 
ism of the category theory, that was invented for the most difficult, apparently 
contradictory aspects of the foundation of mathematics. In this respect we in¬ 
troduced a category of measurable spaces MES. We proved that in the category 
MES exist the direct product and the direct sum, having universal properties. 
We proved that the functional Zp{Mx, px, px) defined in Eq.([T]), which appears 
in the definition of both Renyi and Tsallis entropies, has algebraic compatibility 
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properties with respect to direct product and direct sum, as shown in Eqs (I23p 
and (l28l'). 

The main conclusions may be summarized as follows. 

(1) The natural measure of the quantity of information is the family of function¬ 
als Zp{Mx, fJ-x, Px) given by Eq. ([T]), (defined in the Frechet space for 0 < p < 1, 
and in the Banach space for p > 1), and the classical Shannon entropy by Eq. (|5|) : 

(2) The category MES is the natural framework for treating the problems re¬ 
lated to the measure of the information, in particular in reformulating the Renyi 
axioms; 

(3) The category MES is a monoidal category with respect to direct product 
and coproduct and the functional Zp{Mx, px, Px ) has natural compatibility 
properties with respect to the product (it is multiplicative) and the coproduct 
(it is additive); 

(4) Up to a multiplicative constant, it is possible to recover the exact form of 
the functional Zp{Mx, px, Px) defining the generalized entropies from a system 
of axioms that generalize the ones adopted by Renyi [5]. 
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5 Appendix 

5.1 Renyi divergence and entropy 

Suppose to have a measurable space (12, A, m) with a finite or a -finite measure 
p and a normalized PDF p{x), i.e. f p(x)dp(x) = 1. Only in this Subsection 

n 

we adopt the following definitions 



(41) 


Sr, a (p, dp) = -log U(p, dp, a) 


(42) 


I — a 


Consider now a measurable space N = (12, A, n) with cr-finite measure n. We 
also denote with P{x), Q{x) two probability densities, satisfying the condition 



(43) 


n 

The Renyi divergence reads 



(44) 


n 
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(45) 


According to the notation Eg. (HD) and normalization Ea. (H51) . we get 


J^dn{x) 
n 


U{Q,dn,l - p) 


and Pi{x) := P{x)/U{Q, dn, I — p) is normalized with respect to the measure 
Q^~Pdn{x). Consequently, from Ea. ddTI) . we find 




P 


,Q^ ^dn,p 


ZP 


Z := U{Q, dn, 1 — p) 


(46) 

(47) 


By using the notation Eas lH^ H4E7I) we obtain the following relation between 
the Renyi entropies and the divergences 


Dp{P\\Q) = -SR,p 


Pdn 



Sr,1-P 


[Q,dn] 


(48) 


5.2 Solution of the functional equation Eq. (l37l i 

Using Eq. (IM)) with p > 0, we note that we can use the double logarithmic scale 
by performing the following change of variables 

fx(u) = log hxiexp(u)) (49) 

fY{v) = log/iv(exp(z;)) (50) 

fxviz) = loghxYiexp{z)) (51) 

Hence, Eq. (EZl) reads 

fxYiu + u) = fxiu) + friv) (52) 

From Eq. (15^ . we obtain 

fxYiu + v) - fxYiv) = fviv) - fviO) (53) 

[fxYiu + v) - fxY{v)] - [fxY{u + 0) - fxY{0)] = 0 (54) 

fxY{u + u) - fxY{v) - fxYiu) = /xf(0) (55) 

The Eg. ((5^ admits the particular constant solution 

Fpartiz) = fxYiO) ( 56 ) 

The solution of corresponding homogenous equation 

fxY{u + v) - fxYiv) - fxY{u) = 0 (57) 

may be found by using again the continuity of the function hxYip) (See also 
[20] 1.3.1, page 8), i.e., 

fxY{z)=pz (58) 
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The general solution reads 


fxriz) = /xf(0 ) +pz 

By using Eq. (EH), we get the universal linear slope p 

fx{u) = /x( 0 ) +pu 
friv) = /y( 0 ) +pv 
fxYiO) = fx{0) + fviO) 

and, by Eqs (I49II51I) . up to undetermined multiplicative constants Cx = exp(/x(0)), 
Cy = exp(/y(0)), we find Eqs (I38M0I) . 
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